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ABSTRACT 


The Navy’s Selective Reenlistment Bonus Management System (SRBMS) 
uses a model known as ROGER to identify the SRB-eligible population and to 
predict the number of SRB takers for the following fiscal year. The Enlisted 
Bonus Manager uses the ROGER model to determine the SRB plans during the 
execution year. Over the years, constant changes in the structure of the SRB 
program have led to increasing levels of predictive error in the ROGER model. 
Specifically, the ROGER model has routinely under-identified the SRB-eligible 
population, which, in turn, led to under-predictions in the size of the predicted 
number of SRB takers and the SRB budget. One of the reasons for the under- 
predictions is the ROGER model does not account for sailors who acquire an 


SRB-eligible NEC during the execution year. 


The objective of this thesis is to determine whether the predictive errors in 
the Navy's SRBMS (ROGER) model can be reduced by accounting for new 
NEC/skill acquisition by sailors each fiscal year. NEC/skill acquisitions are 
accounted for by incorporating data into the ROGER model from the Naval 
Education and Training Command (NETC) on annual school enrollments and 
graduations. This thesis analyzes the impact of adding the NETC skill acquisition 
data to the ROGER model by analyzing the predicted SRB-eligible population 
and the predicted number of SRB takers and by assessing the resulting impact 
on the predicted SRB budget. 
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I. INTRODUCTION 


A. BACKGROUND 


The purpose of the Navy’s Selective Reenlistment Bonus (SRB) program 
is to help retain experienced enlisted personnel in critical or undermanned 
occupational specialties. The SRB program helps retention by using selective 
financial incentives to focus retention efforts on those Navy Enlisted 
Classifications (NECs) and ratings that are hard to retain or costly to replace. 
Using targeted SRBs is more cost-effective than giving across-the-board pay 


raises to increase retention. 


The Deputy Chief of Naval Operations (Manpower, Personnel, Training 
and Education) (MTP&E) (N1) has overall responsibility for the Navy’s SRB 
program. The SRB program is considered by N1 to be the “primary monetary 
Force shaping tool to achieve enlisted retention requirements in ratings, Navy 
Enlisted Classifications (NECs) and skills’ (OPNAVINST 1160.8A, 2007). The 
Navy has been using the Selective Reenlistment Bonus Management System 
(SRBMS) to manage the SRB program since 1974. 


1. History 


The Navy’s Selective Reenlistment Bonus Management System (SRBMS) 
uses a model known as ROGER to determine the SRB-eligible population and to 
predict the number of SRB takers for the following fiscal year. The ROGER 
model is a Windows-based application created in the mid-1990s to assist the 
Navy’s Compensation and Policy Coordination Division (N130) and the Enlisted 
Community Managers (BUPERS-32) in determining SRB plans during the 


execution year. 


Over the years, constant changes in the structure of the SRB program 
have led to increasing levels of predictive error in the ROGER model. 


Specifically, the ROGER model has routinely under-identified the SRB-eligible 
1 


population, which, in turn, leads to under-predictions in the size of the predicted 
number of SRB takers (Alloway and Stockton, 2008, p.6). Also, the predicted 
SRB taker population is the main factor used to predict the SRB budget for the 
future fiscal year, thus SRB program costs also tend to be under-predicted: 
Historically, at least since FY97, the Navy has, more often then not, 
failed to contain SRB costs within the congressionally appropriated 
amount. In most years, funds are reprogrammed from within the 


enlisted personnel budget in order to meet the increased costs of 
the SRB program (Alloway and Stockton, 2008, p.1). 


Previous studies have shown the two main causes of the under-prediction 
of SRB takers by the ROGER model for a given fiscal year are due to skill 
acquisition and early reenlistments during the program execution year (Moore, 
Hogan, and Espinosa, 2003, p.5). In FY00, the ROGER model failed to identify 
more than one-fifth of the actual SRB takers as being eligible for an SRB. It was 
determined that 36% of the error between predicted and actual SRB takers was 
due to new skill (NEC) acquisition in the fiscal year (Espinosa, Hogan, and Moore 
2003, p.5). Most of the new NECs are obtained via completion of Department of 
Defense (DoD) technical schools, while a small number are obtained from on- 
the-job training (OJT) or civilian certifications. These new NEC acquisitions 
create hundreds of new SRB-eligible personnel during the execution year who 
are not identified in the initial SRB taker projections. 


B. PURPOSE OF THESIS 


The objective of this thesis is to determine whether the predictive errors in 
the Navy's SRBMS (ROGER) model can be reduced by accounting for new 
NEC/skill acquisition by sailors during the execution year. NEC/skill acquisitions 
are accounted for by incorporating data into the ROGER model provided by the 
Naval Education and Training Command (NETC) on annual school enrollments 
and graduations. This thesis will analyze the impact of adding the new NETC skill 
acquisition data to the predicted SRB-eligible population and the predicted 
number of SRB takers, assessing the resulting impact on the ROGER model’s 


2 


overall prediction error. Reducing the prediction errors in the ROGER model will 
allow for the creation of a more realistic SRB budget, which would greatly reduce 


the need to reprogram funds to cover actual SRB costs. 


C. SCOPE AND METHODOLOGY 


This thesis will determine the effects of NETC school enrollment and 
graduation data upon the ROGER model’s predictions of the SRB-eligible 
population and the number of takers. This thesis will compare the differences 
between the ROGER model’s SRB-eligible population and SRB taker population 
from applying the NETC data vice not using the NETC data. Then, this thesis will 
compare both sets of results, for each fiscal year, on the actual SRB taker 
population for the given fiscal year to see if incorporating the NETC data on skill 
acquisition decreases the prediction error in the ROGER model. In addition, the 
changes to the SRB budget for each fiscal year will be estimated. 


D. ORGANIZATION OF STUDY 


The thesis is organized into the following chapters: 


Chapter Il. Overview of the SRBMS. This chapter gives an overview of 
the SRBMS to include describing eligibility criteria for an SRB, discussing how 
the ROGER model works, and presenting information on the performance of the 
ROGER model from previous studies. 


Chapter Ill. Data and Analytical Methods. This chapter describes the 
data sources for the thesis, the variables used in the analysis, and the analytical 
methods in the ROGER model. 


Chapter IV. Analysis of the Results. This chapter explains the results 
obtained from the analysis and shows the impacts on SRB takers and budget 


projections. 


Chapter V. Conclusion and Recommendations. This chapter 
summarizes the findings from the analysis, states policy recommendations based 


on the analysis, and provides directions for future research. 


ll. OVERVIEW OF THE SELECTIVE REENLISMENT BONUS 
MANAGEMENT SYSTEM (SRBMS) 


This chapter provides an overview of the SRBMS. It includes a 
description of the eligibility criteria for an SRB, a discussion of how the ROGER 
model works, and a discussion of the predictive accuracy of the ROGER from 


previous studies. 


A. SRB PROGRAM ELIGIBILITY 


1. Criteria for Skill Eligibility for an SRB 


The Navy’s SRB program managers determine whether individual ratings 
or NECs are eligible for an SRB, based on the following factors: 

e = Skill or NEC has high training or replacement costs for personnel. 

e Skill or NEC is essential skill to the mission of the Navy. 


e Skill of NEC has severe under-manning in three or more adjacent year 
groups in the bonus zone. 


e Skill or NEC has severe under-manning as shown by a history of 
shortages in past years or projected for future years. 


e Skill or NEC is relatively arduous or unattractive skill compared to the 
other ratings/NECs/skills or civilian alternatives. 


e The SRB award amount improves retentions enough to justify the cost 
(OPNAVINST 1160.8A, 2007, p. 2). 


2. Individual Sailor Eligibility for an SRB 


Individual sailors are determined to be eligible to reenlist for an SRB 
based upon the following criteria: 


e Sailor must be on active duty, except active duty for training, and 
complete at least 17 continuous months of active service, but not more 
than 14 years. 


e Sailor must be eligible to reenlist or extend for at least three years. 


3. 


Sailor has obtained pay grade E-4 or higher, or is an undesignated E-3 
striking for a rating. 


Sailor must be qualified for and serving in an SRB-qualified rating or 
NEC. 


Sailor is not currently receiving entitlement for an SRB, payment of 
readjustment, severance or separation pay. 


Sailor must serve in the rating/NEC/skill for the period of the 
reenlistment for which the SRB was awarded unless the member is 
reassigned to meet the needs of the Navy. 


Sailor must reenlist prior to three months before date of discharge or 
release from active duty service. Director, MPT&E Policy Division 
(N13) may prescribe a lesser period as needed to improve manning 
and retention objectives. 


Sailor must meet any additional criteria prescribed by N13 
(OPNAVINST 1160.8A, 2007, p. 2-3). 


Zone Eligibility Criteria 


As previously stated, SRB eligibility is limited to active duty personnel with 


at least 17 months of continuous, non-training status active duty, but not more 


than 14 years of service. The period of active duty service is broken down into 


three eligibility zones: 


Zone A: The service member must have at least 17 months of 
continuous active military service but not more than six years of 
service on the reenlistment date. 


Zone B: The service member must have completed at least six but not 
more than 10 years of active duty service on the reenlistment date. 


Zone C: The service member must have completed at least 10 but not 
more than 14 years of active duty service on the reenlistment date. 


It should also be noted that only one SRB from each zone can be awarded 
during a career (OPNAVINST 1160.8A, 2007, p. 5). 


B. SRB AWARD AMOUNTS 


Calculation of the applicable SRB award dollar amount is performed using 


the following equation: 


SRB Award = (Monthly base pay * Contract length in years * SRB multiple) 


SRB multiples are offered in increments of .5, with a maximum limit of 15. The 
standard range of multiples is from 0 to 6. As of 30 January 2007, the award 
ceiling for SRB is $90,000 (OPNAVINST 1160.8A, 2007, p. 11). 


C. THE ROGER PREDICTION MODEL 


When attempting to estimate a fiscal year reenlistment prediction scenario 
in the ROGER model, the inputs and calculations are performed in two different 
phases. These two phases are known as the “Pre-Run” phase and the “Run 
Model” phase. Figure 1 diagrams the structure and data flow of the ROGER 
model. 


Pre-Run Run Model 








Source: Mackin, 1999, Appendix B, p. 4. 


Figure 1. ROGER Primary Components 


Before a scenario is run, the “Pre-Run” phase is performed to enter 
multiple primary inputs into the model. These primary inputs include: 

e Consumer Price Index (CPI) deflator 

e SRB lump sum percentage (currently 50%) 


e Pay tables 


e All-Navy Inventory data 
e = Skill modifiers 


e Analysis file (contains Navy enlisted inventory information) 


The “Pre-Run” phase only needs to be opened if it is the first time running the 
scenario or if the data must be updated. (Mackin, 1999, Appendix B, p. 4-5) 


The pay tables and all-Navy inventory inputs must be entered for the initial 
baseline year and for the analysis year. Figure 2 shows the relationship between 
the pay table and all-Navy inventory inputs. The ROGER model condenses the 
information from the pay tables and the all-Navy inventory file into length-of- 
service (LOS) groups. The data from the LOS groups is then used during the 
“bonus level calculation” routine, which makes calculations for the SRB award 
levels by length of reenlistment (3-6 years), length of service (0-12 years), and 
multiplier (0-10). The model then converts the pay information into base year 
dollars, which is added into the initial year’s and analysis year’s ACOL values 
(Mackin, 1999, Appendix B, p. 5). 





Source: Mackin, 1999, Appendix B, p.5. 
Figure 2. Pre-Run Pay and Inventory Inputs 


The user must also update additional “exogenous variables” as necessary 
prior to the Run-Model phase. These variables include: 
e Percentage of SRB paid as a lump sum (currently 50%) 
e Initial year SRB multipliers are entered in the skill modifier table by skill 
and zone 


e Current Consumer Price Index data, which is used to deflate the pay 
and bonuses to base-year values (Mackin, 1999, Appendix B, p. 5). 


When the exogenous variables are updated, the ROGER model then 
determines the changes in reenlistment probabilities among all skill and LOS 
groups, based on changes made to the initial SRB multiplier plan. This new 
probability data is stored in the “delta reenlistment probability matrix,” which 
allows running of new scenarios without recalculating reenlistment probabilities 
each time (Mackin, 1999, Appendix B, p. 5). Figure 3 diagrams the flow of the 


exogenous variables. 





Source: Mackin, 1999, Appendix B, p. 6. 
Figure 3. Additional Pre-Run Inputs 


In the last portion of the Pre-Run phase, illustrated in Figure 4, the 
analysis file is used to construct the initial reenlistment rates, identify the 
inventory of SRB-eligible personnel, and predict quarterly reenlistments. The 
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“eligible inventory file” is multiplied by the reenlistment probability, from the 
reenlistment probability matrix, to obtain the predicted reenlistments in a given 


fiscal year by quarters or for an entire year (Mackin, 1999, Appendix B, p. 6). 





Source: Mackin, 1999, Appendix B, p. 6. 


Figure 4. Analysis File Data Generation 


Once the Pre-Run phase has been completed, the user starts the “Run” 
phase by inputting the analysis year information into the “run scenario” and 
“scenario settings” screens. The user can then enter the planned SRB 
multipliers for each skill and zone, the award ceilings (currently $90,000) and 
“fenced” (buffered from alterations in multiplier levels) skill groups. The 
adjustable scenario settings include: 


e the maximum multiplier level change between the base year and 
analysis year, 


e the length of time for which the analysis should be run (one year or 
one-three quarters) 


e the maximum bonus level to be used (Mackin, 1999, Appendix B, p. 7). 
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As illustrated in Figure 5, reenlistment probabilities associated with the 
given SRB plan are applied to the eligible inventories that were identified in the 
“Pre-Run” phase. The model then generates predicted reenlistments (by skill, 
length of service, length of reenlistment and EAOS), the costs of the SRB plan, 
and finally a report showing the percentage of takers above the critical value 
bonus level. The enlisted community managers previously required the creation 
of the critical value bonus report to determine the number of personnel reenlisting 
for SRBs above a certain bonus multiplier. This report aided in shaping of the 
SRB multiplier list for each fiscal year. The results of the ROGER model 
predictions can be displayed by: 

e Bonus plan cost 


e Reenlistment rate 


e Number of reenlistments (Mackin, 1999, Appendix B, p. 7). 








Source: Mackin, 1999, Appendix B, p. 7. 
Figure 5. Run Model Structure 


D. MODEL ASSESSMENT 


is Prediction History 


Alloway and Stockton (2008, p.1) report that, “Historically, at least since 
FY97, the Navy has, more often then not, failed to contain SRB costs within the 
congressionally appropriated amount. In most years, funds are reprogrammed 
from within the enlisted personnel budget in order to meet the increased costs of 
the SRB program.” The requested and actual SRB program expenditures are 


shown in Table 1 and Figure 6. 


Table 1 shows the differences between the requested SRB budget per 
fiscal year (column 1) and the actual expenditures (column 2). The missed 
percentages in column 4 are based on comparison of the total amount the 
budget missed as a ratio of the actual SRB program expenditure for each fiscal 
year. Table 1 shows the statistics numerically while Figure 6 displays the 


requested and actual SRB expenditures (in millions of dollars) each year. 


Table 1. Navy’s Selective Reenlistment Bonus Program’s Requested and Actual 
Expenditures for FY97-07 


(In $ Millions) 
Requested Actual Actual - Requested | Missed (%) 


FY 1997 
FY 1998 
FY 1999 
FY 2000 
FY 2001 
FY 2002 
FY 2003 
FY 2004 
FY 2005 
FY 2006 
FY 2007 















































Source: Navy budget justification books FY97-09 


“ FY2007 actual expenditures were as of Feb 08 and, therefore, are incomplete 
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Source: Navy budget justification books FY97-09 


Figure 6. Navy’s Selective Reenlistment Bonus Program’s Requested and 
Actual Expenditures for FY97-07 (In $ Millions) 


2. Lewin Group Study 


In 2003, the Lewin Group (Moore, Hogan and Espinosa, 2003) was asked 
by the Navy to perform an independent testing and analysis of the accuracy of 
the SRBMS. The Lewin Group’s analysis consisted of testing the key 
components of the model as well as the model as a whole. For their analysis, 
the Lewin Group looked specifically at predictions of several outcomes for FY00: 

e The SRB-eligible population 

e The number of personnel who reenlist for an SRB (SRB takers) 

e The average length of reenlistment among the SRB takers 


a. SRB-eligible Population 


According to the Lewin Group study, the ROGER model under- 
predicts the actual size of the SRB-eligible population. For FY00, over one-fifth 
of the 15,048 actual SRB takers were not identified as SRB-eligible. The model 
predicted most accurately in Zone A, with 82% of the SRB takers predicted to be 
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eligible, while in Zones B and C only 70% of the SRB takers were identified as 
eligible. ©The reasons for the under-identification of SRB-eligibles was 
ascertained to be caused by two primary factors: new skill acquisition by sailors 
during the fiscal year, and early reenlistments (Moore, et al., 2003, p.11). 


For FY00, 36% of the 3,402 actual takers who were not identified 
as being SRB-eligible were mis-identified because they obtained a new SRB- 
eligible skill during the planning fiscal year. Skill acquisition caused 68% of the 
error in Zone A, 24% of the error in Zone B and 8% of the error in Zone C. Skill 
acquisition had the greatest impact upon predictions for the HM rating, with up to 
93.5% of missing eligibles in the HM rating being caused by new skill acquisition 


during the fiscal year (Moore, et al., 2003, p.14-15). 


The second factor affecting identification of the number of SRB- 
eligibles is the number of sailors who reenlist for an SRB prior to the time they 
are in the normal reenlistment eligibility window. The ROGER model only selects 
as SRB-eligible those personnel who are within 13 months of their EAOS and 
who hold an SRB granting NEC. The Lewin Group identified three main reasons 
for early reenlistment for an SRB outside of the normal eligibility window: 


1) Permanent Change of Station (PCS) orders 
2) Sailors holding Nuclear NECs 


3) Sailors holding submariner ratings/skills 
All three factors allow personnel to reenlist for an SRB prior to the normal 
reenlistment eligibility window, which makes it impossible for the ROGER model 
to account for these personnel in each fiscal years pool of SRB-eligible 
personnel. While these early reenlistment reasons can help with identifying 
eligibles, they still accounted for only about one-half (47%) of the 2,215 
personnel who reenlisted early (Moore, et al., 2003, p.16). 


b. SRB Takers 


According to the Lewin Group study, the ROGER model under- 
predicts actual reenlistments in all of the SRB zones. The Lewin Group realized 
the error from the eligibles pool would have an effect upon SRB taker predictions, 
so they decided to remove from their analysis file any actual takers who were not 
identified as eligible (Moore, et al., 2003, p. 18-19). 


Table 2 displays the differences between the predicted and actual 
SRB takers for each zone in FY0O. For FYOO, the average difference between 
the predicted and actual SRB takers was 14.5%. Zone B had a prediction error 


of 11.9%, while Zone C had an error of 22%. 


Table 2. Actual and Predicted SRB Takers, by Zone, for FYOO 


(3) (4) 
(1) (2) Difference Percent Difference 
Zones Predicted Actual =(1-2) (%) 
Zone A 6,489 7,493 1,004 14.4% 


2,584 2.911 327 
| Zonec | 996 | 1,242 246 
10,069 11,646 1,581 


Note: The percentage difference uses the average of actual and predicted values as its base: 
=(1-2)/((1+2)/2) Source: Moore, et al., 2003, p. 19 








Table 3 shows the Lewin Group’s predicted and actual 
reenlistments broken down by zones and occupation groups for FY00. The table 
shows the disparity in predictions across the broad occupation groups and 
zones. Some occupations, such as Non-electronic maintenance, Aviation 
maintenance, and Logistics, have over-predictions in certain zones, while other 
occupations have severe under-predictions. As shown in Table 3, reenlistments 
for Electronic maintenance occupation group were under-predicted by 528 
personnel in FYOO in all three zones. 


Table 4 shows the breakdown of under-predicted, over-predicted, 
and exactly predicted reenlistments by zone and skill groups for FYOO. 
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Table 3. Predicted and Actual Reenlistments by Zone and Occupation Group, for 
FYOO 
Zone A Zone B Zone C 
Occupation 
Group Predicted | Actual | %Diff | Predicted | Actual | %Diff | Predicted | Actual | %Diff 
Non- 
electronic 1,704 1,867 | 9.1% 340 316 | -7.3% 191 173 -9.9% 
maintenance 
Seoul 1,776 | 2,304 | 25.9% | 584 699 | 17.9% | 355 361 | 1.7% 
maintenance 
Aviation - : 9 
anes 643 570 | 10.9% 201 147 | 34 09% 1 9 160.0% 
Ship/Aviation | 4576 | 1,362 | 6.5% 542 686 | 23.5% 42 150 | 112.5% 
support 
Health care 122 141 | 14.4% 240 242 | 0.8% 28 34 19.4% 
Logistics 208 263 | 23.4% 103 79 26.4% - - NA 
Construction 253 353 | 33.0% 207 266 | 24.9% 1 1 0.0% 
Cryptology 361 395 | 9.0% 161 188 | 15.5% 60 104 | 53.7% 
Admin, 
media and 146 238 =| 47.9% 206 288 | 33.2% 318 410 | 25.3% 
other 
Note: The percentage differences use the average of actual and predicted values as its base 
Source: Moore, et al., 2003, p. 21 
Table 4. Number of SRB Skill Groups Contributing to Prediction Error 
Zone A Zone B Zone C 
Under-predict reenlistments 106 96 47 
Over-predict reenlistments 54 67 18 
Exactly predict reenlistments 91 79 36 
Total 251 242 101 




















Source: Moore, et al., 2003, p. 22 
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3. Alloway and Stockton Study (2008) 


In FY04, changes were made to the algorithm in the ROGER model that 
allowed it to look at all of the NECs that each individual holds when determining 
eligibility for an SRB. When multiple NECs are present, the model now selects 
the NEC that offers the largest bonus. Prior to FY04, the ROGER model only 
looked at the Primary NEC (PNEC) of each individual. 


Alloway and Stockton performed an analysis on the updated ROGER 
model to see how the changes to the algorithm affected model predictions. 
Unlike the Lewin Group study, Alloway and Stockton used multiple fiscal years 
(FY04-07) in their study to try to account for any relevant trends (Alloway and 
Stockton, 2008, p. 57). 


In their study, Alloway and Stockton found that the model did not correctly 
identify between 21% and 27.9% of the actual takers as eligible during the FY04- 
07 period. While the total number of under-predictions varied across year and 
zone, the rankings of percentage missed by zone stayed consistent, with Zone A 
having the lowest percentage of takers who were not predicted as eligible, and 
Zone C having the highest percentage of takers who were not predicted as 
eligible (Alloway, et al., 2008, p. 61-64). Tables 5 through 8 show the differences 
between actual takers and the predicted eligibles for each fiscal year during the 
FY04-FY07 period. 


Table 5. Alloway and Stockton, FY04 Snapshot of Number of Eligibles, Takers, 
Mis-Predicted Takers, and Percentage Error 


(1) (2) (3) 
Total SRB Identified SRB Missed SRB 
Takers Reenlisters Reenlisters 
=(1-2) 





7,672 6,133 1,539 























Source: Alloway and Stockton, 2008, p. 6 


Table 6. 


Alloway and Stockton, FY05 Snapshot of Number of Eligibles, Takers, 


Mis-Predicted Takers, and Percentage Error 


(1) 
Total SRB 
Takers 


(2) 
Identified SRB 
Reenlisters 


(3) 
Missed SRB 
Reenlisters 

=(1-2) 


(4) 
Percent 
Error 





10,194 


8,933 


1,661 


16.3% 





4,589 


3,295 


1,334 


29.1% 





1,062 











Source: Alloway and Stockton, 2008, p.62 


Table 7. 








4,057 





Alloway and Stockton, FY06 Snapshot of Number of Eligibles, Takers, 


Mis-Predicted Takers, and Percentage Error 


(1) 
Total SRB 
Takers 


(2) 
Identified SRB 
Reenlisters 


(3) 
Missed SRB 
Reenlisters 

=(1-2) 


(4) 
Percent 
Error 














Source: Alloway and Stockton, 2008, p.63 


Table 8. 











Alloway and Stockton, FY07 Snapshot of Number of Eligibles, Takers, 


Mis-Predicted Takers, and Percentage Error 


(1) 
Total SRB 
Takers 


(2) 
Identified SRB 
Reenlisters 


(3) 
Missed SRB 
Reenlisters 

=(1-2) 


(4) 


Percent Error 





6,385 


5,443 


942 





4,622 


3,054 


1,568 





2,407 


1,184 


1,223 








13,414 





9,681 


Source: Alloway and Stockton, 2008, p. 64 
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3,733 








Alloway and Stockton also performed an analysis to determine the number 
of takers who were not originally identified as eligible but who had obtained an 
SRB-eligible skill or NEC during the planning fiscal year. Alloway and Stockton 
focused on the FY07 data for this analysis. The analysis showed that 12% of the 
non-identified eligibles were due to new skill acquisition during the fiscal year. 
This was a big drop from the 25% identified by the Lewin Group study in FYOO. 
Alloway and Stockton attributed the difference between FY00 and FYO7 to the 
changes made to the ROGER model’s algorithm in FY04 and to the unique 
eligible pools for the given fiscal years. In addition, 86% of the missing takers 
were outside the eligibility window. Sailors are within the reenlistment eligibility 
window if they are within 13 months of their end of obligated service (EAOS). Of 
those outside the eligibility window, 13% were in Zone A and 73% were in Zone 
B or C (Alloway and Stockton, 2008, p.65-66). 

While the overall analysis statistics explaining the ROGER model 
prediction errors varied between the Lewin Group study and the Alloway and 
Stockton study, Alloway and Stockton still identified the two main reasons for 
mis-identification of takers as: 

e sailors have gained a new NEC during the fiscal year, and 


e reenlistments are outside of the eligibility window (i.e., reenlistment 
due to PCS orders or NUC NECs) 


Alloway and Stockton also attributed a number of other potential reasons for the 
poor projections of the SRB takers, to include: 

e Improper establishment of baseline reenlistment rates 

e Inaccurate pay elasticities 

e Poor grouping of Navy skill groups (Ratings/NECS) 

e Changes in civilian unemployment rates 

e Missing or deficient data 


e Unforeseen events affecting behavior (e.g., increased deployments to 
tax-free areas) (Alloway and Stockton, 2008, p.77). 


E. CHAPTER SUMMARY 


The ROGER model is designed to run in two phases: pre-run and run. 
During the pre-run phase, users enter the primary data inputs and use the model 
to construct the initial reenlistment rates and identify the number of SRB-eligible 
sailors. Once the pre-run phase is complete, the run phase is started and the 
user inputs the SRB multipliers, length of time the analysis should be run, and 
the maximum bonus level. The scenario is then set to run and the ROGER 
model projects the number of reenlistments by NEC and the cost of the SRB 


bonus plan. 


Multiple studies over recent years have shown that, on average, the 
ROGER model fails to correctly identify approximately 25% of the actual SRB 
takers as SRB-eligible. This was first determined by the Lewin Group study in 
2003 and then reconfirmed by the Alloway and Stockton study in 2008. Both of 
these studies determined that the main factors causing the under-identification of 
eligibles were: 1) sailors gaining new SRB-eligible skills/NECs during the fiscal 
year, and 2) sailors reenlisting outside of the normal eligibility window. 


While there are multiple factors causing under-identification of eligibles in 
the ROGER model, this thesis will look specifically at the under-identification of 
eligibles due to sailor acquisition of new SRB-eligible skills/NECs during the fiscal 
year. This thesis will determine whether the predictive errors in the Navy’s 
SRBMS model can be reduced by accounting for sailor NEC/skill acquisition 
each fiscal year, incorporating Naval Education and Training Command (NETC) 
annual school enrollment and graduation data into the ROGER model. 
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lll. DATA AND METHODOLOGY 


This chapter describes the data sources used in this thesis and discusses 
the variables used in the analysis and the methods of analysis used to determine 
the effects of incorporating the execution-year school enrollment data into the 
ROGER model. 


A. DATA 


1. Data Sources 


The source of enlisted school enrollment and graduation data for this 
analysis was the Navy’s Corporate enterprise Training Activity Resource System 
(CeTARS) database, which is controlled by Naval Education and Training 
Command (NETC). CeTARS is the Navy’s database for training course 
description and _ statistical information on all formal training courses. 
OPNAVINST 1510.10B requires information from every formal course of 
instruction conducted in the Navy to be reported to CeTARS. Formal training 
courses that are reported to CeTARS include accession training, initial and 
general skill training, specialized skill training, and career development training 
that focuses on training for a designated billet or specialty (OPNAVINST 
1510.10B, 2005). The NETC Business Operations Branch (N63) is the 
designated CeTARS program manager. All requests for CeTARS data are 
routed through N63. 


The ROGER model uses a snapshot of the Navy Enlisted Master Record 
(EMR) to perform its initial determination of the SRB-eligible population. The 
SAG Corporation provided up-to-date EMR snapshots along with a copy of the 
ROGER model executable files. 
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2. Data Extraction and Compilation 


For this analysis, the planned enrollment and graduation data was 
extracted and compiled for FY04 to FY08. To extract the specific NEC 
enrollment and graduation data from the CeTARS database, the Course Data 
Processing (CDP) codes were used. These codes are associated with each 
NEC in the Manual of Navy Enlisted Manpower and Personnel Classifications 
and Occupational Standards, NAVPERS 18068F. The CDP is the code assigned 
to uniquely identify each course of instruction and its physical location. If the 
same course is taught at different physical locations, it has different CDP codes. 


Some of the CDPs have NECs associated with them in the CeTARS data 
files, but analysis of the CeTARS database files showed that many of the CDPs 
that have NECs associated with them in NAVPERS 18068F did not have any 
NEC information assigned to them in the CeTARS data files. This random 
assignment of NECs to the corresponding CDPs in the CeTARS database 
prevented easy compilation of the NEC data. The total graduates for each NEC 
had to be compiled by manually totaling the graduates for all of the CDPs that 
award each NEC. 


B. METHODOLOGY 


1. Data Input 


While the compiled dataset contained planned enrollment and graduation 
data on 116 NECs for each fiscal year, only 16 of these NECs from FY04 to 
FY08 were used in the analysis. These NECs were used to allow for an initial 
analysis of incorporating the new CeTARS data into the ROGER model, but to 
avoid the heavy time burden required to manually input the entire data set into 
the EMR snapshots. To choose which NECs were to be used in this analysis, 
the actual SRB taker file for FYO7 was analyzed; those NECs that had at least 50 
actual SRB takers during the fiscal year, or that historically had poor prediction 


performance, were chosen for the analysis. The NECs with more than 50 actual 
22 


SRB takers were chosen to reduce the number of overstated effects due to small 
changes in the predicted SRB takers from incorporating the CeTARS data. 
Some of the NECs have small amounts (less than 10) of actual takers each fiscal 
year, so using these NECs would show dramatic effects from very small 
increases in the number of predicted takers. The NECs for the HM rating were 
included because they historically have had poor prediction performance in the 
ROGER model. The NECs chosen for the analysis were also spread across 
multiple occupation groups. Table 9 lists the NECs used in the analysis along 
with their title and associated community. The table shows that the NECs are 


spread across the Surface, Intel, Specwar, Medical, and Cryptology 






































communities. 
Table 9. List of NECs Used for Analysis 
SRB 
NEC DESCRIPTOR TITLE COMMUNITY 
2735 | IT-SYS ADMIN Information Systems Administrator Surface 
2780 | IT-NET SEC Network Security Vulnerability Technician Surface 
2781 IT-ADV NET Advanced Network Analyst Surface 
3910 | IS-IMAGE Naval Imagery Interpreter Intel 
3924 | IS-NTCS Operational Intelligence (OPINTEL) Analyst Intel 
5326 | SO-SEAL Special Warfare Operator (SEAL) SpecWar 
5343 | ND-SECOND Diver Second Class SpecWar 
Special Warfare Combatant Crewman (SWCC) 
5352 | SB-SWCC AD Advanced SpecWar 
8425 | HM-SW IDC Surface Force Independent Duty Corpsman Medical 
8432 HM-PREV Preventive Medicine Technician Medical 
8452 HM-XRAY AD Advanced X-ray Technician Medical 
8482 HM-PHARM Pharmacy Technician Medical 
8483 HM-SURG Surgical Technologist Medical 
8506 HM-MED TEC Medical Laboratory Technician Advanced Medical 
9138 | CTR-A&R Journeyman Analysis and Reporting Specialist Crypto 
9147 | CTR-INTSIG Intermediate Signals Analyst Crypto 




















Source: Author created 


The data was incorporated into the EMR snapshot by assigning an NEC to 
an individual that met the requirements for the NEC (rank, rating, LOS, 
prerequisite NECs) but did not already have the NEC in the EMR snapshot. The 
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graduates for each fiscal year were assigned across the paygrades of each NEC 
based on the ratios of the paygrades for that NEC in the list of actual takers for 


the given fiscal year. 


The CeTARS data was incorporated into the EMR snapshot using the 
following data compilations: 


e The actual number of graduates (average) for FY06 thru FY08 in each 
NEC 


e The planned graduates for FY07 and FY08, for each NEC, calculated 
via the following formula: (Previous 2-Yr Actual Grads / Previous 2-Yr 
Planned Enroll) x Current FY Planned Enrollment 


This formula determines the ratio of graduates to planned enrollment over the 
two previous years, and then multiplies this ratio by the given fiscal year’s 
planned enrollment to determine the number of planned graduates for a given 
fiscal year. The purpose of using the number of planned graduates is to try to 
account for changes in the total actual graduates in a fiscal year due to 
fluctuations in the planned enrollment of each NEC. The fiscal year enrollment 
plans tend to have erratic variations across the NECs and skill groups from year 
to year, so it is hypothesized that using the actual planned number of graduates 
in the modified databases should work better than just using the graduate 
averages for the previous two fiscal years. 


The actual graduate average data is included in the ROGER model to 
determine their impact on the ROGER model’s prediction of SRB takers. The 
prediction results using the two modified databases are compared to the baseline 
predictions as well as to each other to see if there is a notable difference in the 
predictions from using the average number of actual graduates or the calculated 
number of planned graduates. 


Table 10 shows the planned enrollment numbers by NEC for FY 2006 to 
FY 2008. As one can see, the enrollment plans for some NECs are relatively 


constant over time, while others have huge increases or decreases from year to 


24 


year. Most of these fluctuations are due to the enlisted community managers 
adjusting school quotas to try to manage the size and rank structure of their 


given communities. 


Table 10. Planned Enrollments by NEC and Fiscal Year 


474 


Source: Author created from CeTARS database 





Table 11 shows the calculated planned number of graduates by NEC for 
fiscal years 2007 and 2008. These are the numbers used to assign new NECs to 
individuals in the modified databases using NEC calculated planned graduates. 
Some of the calculated planned graduate numbers stay relatively consistent 


across the NEC from year-to-year, while others fluctuate by a few hundred. 
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Table 11. 


Descriptor 
IT-SYS AD 


COMMUNITY 
Surface 


Calculated 
Planned 
Graduates 
FY2007 


Calculated Number of Planned Graduates by NEC, FY07-FY08 


Calculated 
Planned 
Graduates 
FY2008 





IT-NET SEC 


Surface 





IT-ADV NET 


Surface 





IS-IMAGE 


Intel 





IS-NTCS 
SO-SEAL 
ND-SECOND 


Intel 


SpecWar 











SB-SWCC AD SpecWar 
HM-SW IDC Medical 
HM-PREV Medical 
HM-XRAY AD Medical 
HM-PHARM Medical 
HM-SURG Medical 
HM-MED TEC Medical 
CTR-A&R Crypto 
CTR-INTSIG Crypto 
Source: Author created from CeTARS database 









































Table 12 shows the prior two-year average number of graduates by NEC 
for fiscal years 2006 through 2008. These are the numbers used to assign new 
NECs to individuals in the modified databases based on the average number of 
graduates. As with the calculated planned graduates, some of the average 
graduate numbers stay relatively consistent across years, while others go up or 
down by a couple hundred. Again, most of these fluctuations are due to the 
enlisted community managers adjusting school quotas to try to manage the size 


and rank structure of their given communities. 
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Table 12. Prior Two-year Average Number of Graduates by NEC, FY06-FY08 


FY06 Prior Two- | FY07 Prior Two- FY08 Prior Two- 
year Average year Average year Average 
Descriptor Graduates Graduates Graduates 
IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 
ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 
HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author created from CeTARS database 
























































2. Analysis Methods 


The analysis portion of this thesis focuses on comparing the differences in 
outputs from the ROGER model while using the unmodified EMR snapshot 
versus using the EMR snapshot modified with the CeTARS NEC enrollment data. 
From here on, the output from using the unmodified EMR snapshot in the 
ROGER model is referred to as the “baseline” output while the output from using 
the EMR snapshot incorporated with NETC CeTARS data is referred to as the 
“modified” output. 


For this analysis, the ROGER model prediction outputs from using the 
FY06 to FYO8 EMR snapshots are analyzed using the following five focus points: 


e First, the baseline eligibles pool created from the unmodified EMR 
snapshot is compared to the eligibles pool created from the modified 
EMR snapshot to see how the newly assigned NECs affect the 
predicted number of eligibles 


2/ 


Second, the baseline predicted number of SRB takers is compared to 
the modified predicted SRB takers per fiscal year to determine the 
effect of the modified databases on the number of predicted takers 


Third, the baseline and modified SRB taker predictions are compared 
to the actual number of SRB takers per fiscal year to see if the 
modified EMR snapshot reduced the error between the predicted and 
actual number of SRB takers 


Fourth, the impact of the modified databases on the predicted SRB 
budget is analyzed 


Fifth, the modified databases effects upon the number of predicted 
takers in ROGER is compared to the current practice of increasing the 
ROGER model taker predictions by a certain percentage 


The results for the five focus points are explained in Chapter IV. 
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IV. ANALYSIS 


This chapter will display and discuss the results of the analysis performed 
on the ROGER model outputs using the baseline database and the databases 
modified with the CeTARS graduate data. The analysis will include presentation 
and discussion of the eligibles pools, a comparison of the number of predicted 
and actual SRB-takers, the budget effects from using the modified databases, 
and a discussion of other options for reducing the mis-predicted takers. 


A. ELIGIBLES ANALYSIS 


The first output created by the ROGER model is the pool of SRB-eligible 
personnel created by analyzing the Enlisted Master Record (EMR) snapshot. 
This analysis will compare all of the eligible pools that were created using the 
baseline and modified databases. 


1. Comparison of Eligibles Output 


Table 13 and Table 14 compare the total “baseline” eligibles output with 
the eligibles output created from using the “modified” databases. As the data 
shows, incorporating the previous two-year average graduate data into the 
database increased the number of eligibles by 25%—27% in Table 13. Using the 
calculated planned graduate data increased the number of eligibles by 26%—27% 
in Table 14. The changes in eligibles by individual NEC are shown in Tables 15 
through 17. 
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Table 13. Comparison of Predicted Number of Eligibles Using Modified Average 
Graduates to Baseline EMR 


Eligibles Using Percentage 
Baseline Average Increase in 
Graduates Difference igi 























Source: Author created from ROGER model output 


Table 14. Comparison of Predicted Number of Eligibles Using Modified Planned 
Graduates to Baseline EMR 


Eligibles Using Percentage 
Baseline Planned Increase in 
FY Eligibles Graduates Difference Eligibles 
662 





2007 2426 3088 27.3 % 
2008 2603 3299 696 26.7 


Source: Author created from ROGER model output 





Tables 15-17 show the effects of using the “modified” database on the 
predicted number of eligibles, by NEC, for FY06 to FY08. The data indicates 
wide variations in the effects of assigning the new NECs into the modified 


databases. Some NECs had large percentage increases in the predicted 


number of eligibles, while others had little to no change. However, the 


percentage increases are often large because they are calculated on a very 


small base. The percentage increase in the number of eligibles ranged from 0 to 


555%, with 25% of the NECs having increases of the number of eligibles of less 


than 10%, 48% of the NECs having eligibles increases of less than 30%, and 
76% of the NECs having eligibles increases of less than 50%. NEC 8425 is a 
noticeable outlier because the percentage increase averaged around 500%. 


This large percentage increase was primarily due to the low baseline predictions 
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of 8 to 12 predicted takers. While the percentage increase in eligibles was high 
for some NECs, none of the NECs had large increases in their number of 


eligibles, with the range being 0 to 86. 


Table 15. Comparison of Number of Eligibles by NEC Using Modified Average 
Graduates to Baseline EMR, FY06 


Eligibles 
Increase Using Percentage 
FY06 Baseline Average Increase In 
Descriptor Eligibles Graduates Eligibles 
IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 
ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 
HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author Created from ROGER model output 
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Table 16. 


Comparison of Number of Eligibles by NEC Using Modified Databases to 
Baseline EMR, FY07 


Descriptor 
IT-SYS AD 


FY07 
Baseline 
Eligibles 


Eligibles 
Increase 
Using Average 
Graduates 


Percent 
Increase 
Using 
Average 
Graduates 


Eligibles 
Increase 
Using 
Planned 
Graduates 


Percent 
Increase 
Using 
Planned 
Graduates 





IT-NET SEC 





IT-ADV NET 





IS-IMAGE 





IS-NTCS 





SO-SEAL 





ND-SECOND 





SB-SWCC AD 





HM-SW IDC 





HM-PREV 





HM-XRAY AD 





HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 








CTR-INTSIG 








Source: Author Created from ROGER model output 
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Table 17. Comparison of Number of Eligibles by NEC Using Modified Databases to 
Baseline EMR, FY08 


Percent Eligibles Percent 
Eligibles Increase Increase Increase 
FY08 Increase Using Using Using Using 
Baseline Average Average Planned Planned 
Descriptor Eligibles Graduates Graduates | Graduates | Graduates 


IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 




















ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 
HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author Created from ROGER model output 












































2. Comparison of NEC Input to SRB Eligibility Selection 


While it is important to know the change in the overall number of SRB 
eligibles, comparing the increase in eligibles to the number of NECs assigned in 
the database is also important to understand the total effects of incorporating the 
graduate data. Tables 18 and 19 show the total number of personnel assigned 
an NEC in the modified databases and the percentage of those personnel having 
a newly assigned NEC that were selected as eligible by the ROGER model. As 
seen in Tables 18 and 19, there is a large difference between the number of 
personnel assigned a new NEC in the modified databases and the ROGER 
model’s eligibility outputs using those modified databases. One example is FY06 


33 


in Table 15, where 2,895 new NECs were assigned, but the ROGER model 
selected only 584 of these personnel as eligible in Table 18, which was roughly 
20% of the new NECs assigned. 


One of the reasons for the difference between the actual number of NECs 
assigned and the increase in the number of eligibles in the ROGER model is that 
when the databases were created the NECs were assigned randomly across the 
database to those personnel who met the minimum requirements for the NEC. 
The minimum requirements for assigning an NEC in the databases were set as 
the requirements for obtaining the NEC in NAVPERS 18068F. These 


requirements were specific to rating, rank and time in service. 


For the ROGER model to select all of the new NEC-assigned personnel 
as SRB-eligible, the NECs would have had to be assigned only to those 
personnel who were within the reenlistment window of 13 months from their 
EAOS. Since there have been no previous studies to determine the rate of 
personnel obtaining an NEC and reenlisting in the same year, it was believed, by 
members of the SAG Corporation as well as the author, that randomly assigning 
the NECs across the pool of individuals who meet NEC minimum requirements 
would be more realistic. The number of previous two-year average actual 
graduates and calculated planned graduates in any given fiscal year was very 
similar to the number of baseline eligibles, so it is believed that setting all of the 
graduates as eligible in the database would have doubled the number of eligibles 
and most likely skewed the analysis data to the high side. Because the actual 
percentage of personnel who obtain an NEC and reenlist for an SRB in the same 
fiscal year is not known, the NECs were assigned randomly across the 
databases instead of specifically being assigned to only those who would be 
eligible. 
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Table 18. Percentage of Personnel Assigned an NEC that were Selected as Eligible 
in ROGER (Using Average Number of Graduates) 


New NEC Assigned Total Percentage of 
Using Average New NEC Selected as 
Difference Graduates Eligible in ROGER 


584 2895 20.2% 
610 3352 18.2 
688 3216 21.4 
Source: Author Created from ROGER model output 




















Table 19. Percentage of Personnel Assigned an NEC that were Selected as Eligible 
in ROGER (Using Planned Number of Graduates) 


New NEC Assigned Total Percentage of 
Using Planned New NEC Selected as 
FY Difference Graduates Eligible in ROGER 


2007 662 3376 19.6% 
2008 696 3085 22.6 
Source: Author Created from ROGER model output 








Table 20 displays the percentage of new NEC holders selected as eligible 
in ROGER using the modified databases. Table 20 shows that the percentage 
of new NEC holders selected as eligible varied greatly across NECs. The 
percentage of new NEC holders who were selected as eligible ranged from 0 to 
71%, with the averages for the fiscal years ranging from 20.8%-27%. This 
variation is most likely due to differences in the number of personnel to which the 
new NECs were assigned. Certain NECs are assigned to multiple ratings, while 
others are assigned to only one rating or have strict prerequisites for obtaining 
the NEC. An example of this is NEC 8425 HM-SW IDC and NEC 5351 SB- 
SWCC IN. NEC 8425 only has the requirement of being an HM in pay-grade E5 
to E9, where NEC 5351 has multiple requirements to obtain the NEC. Having 
more restrictions for the NEC would greatly reduce the number of personnel who 


are available in a given year group for reenlistment. 
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Table 20. New NEC Holders Selected as Eligible in ROGER Model (In Percent) 


FY06 FY2007 FY07 FY08 Average 
Average Average Planned Average Across 
NEC Descriptor | Graduates | Graduates | Graduates | Graduates | Graduates NEC 


ae 
11.56 
18.52 
ac 

2 | 1482 
r5326[So-SEAL | 303 | 618 [| 632 | 69 574 

10.08 
rs362[se-swoc aD] o | 4se | esr | 26 3.14 

38.97 
12.39 
57.72 
37.49 
ae 
38.8 

19.75 
12.05 
24.6 


Source: Author Created from ROGER model output 





B. ROGER MODEL PREDICTIONS OF SRB TAKERS 


The second output created by the ROGER model is the predicted number 
of SRB takers. This section of the analysis will look at the difference between the 
baseline taker predictions and the taker predictions from the “modified” 
databases. Those differences will be compared to the actual takers for each 
fiscal year to determine the reduction in prediction error from using the modified 
databases. 


1. Comparison of SRB Taker Predictions from ROGER Model 
Based on Baseline, Average Graduates, and Planned 
Graduates 


Tables 21—23 show the actual change in the number of predicted takers 


using the modified databases, and Table 24 shows the percentage change in 
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predicted takers from the baseline when using the modified databases. As with 
the changes in the eligible pools, the change in the ROGER model’s SRB taker 
predictions varied across NECs. In addition, an important item to note is the data 
shows that there is no determinable constant difference between predictions 
based on the calculated number of planned graduates and the number of 
graduates using the previous two-year average. It was hypothesized that the 
calculated planned graduate numbers would work better than the average 
number of graduates at reducing the prediction error, but the data in Tables 21- 
23 show the effects to be very similar. 


Looking at Tables 21—23, none of the NECs showed large increases in the 
number of predicted takers. The highest increase in predicted number of takers 
was 27 for NEC 2780 in FY08, which represented 21% of the baseline predicted 
takers. While the total increase in predicted takers were not very high, some of 
the NECs had large percentage increases in predicted takers from the baseline 
numbers. The overall percentage increase in the predicted takers was the 
highest for the HM NEC 8425 (with 300% to 400% increases), but the baseline 
predicted takers for this NEC was also extremely low each fiscal year (ranging 
from 4 to 6 predicted takers). 
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Table 21. ROGER Model Predicted Number of SRB Takers, FY06 


Increase in Predicted 
Baseline Predicted | Predicted Takers Takers Using Average 
Takers Using Average Graduates 

Descriptor (1) Graduates (2) =(2-1) 


IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 
ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 


HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author Created from ROGER model output 
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Table 22. 


Descriptor 
IT-SYS AD 


Baseline 
Predicted 


Average 
Graduates 
Predicted 
Takers (2) 


Planned 
Graduates 
Predicted 
Takers (3) 


Average 
Graduates 
Change from 
Baseline 
(2-1) 


ROGER Model Predicted Number of SRB Takers, FY07 


Planned 
Graduates 
Change from 
Baseline 
(3-1) 





IT-NET SEC 





IT-ADV NET 





IS-IMAGE 





IS-NTCS 





SO-SEAL 





ND-SECOND 





SB-SWCC AD 
HM-SW IDC 








HM-PREV 
HM-XRAY AD 








HM-PHARM 





HM-SURG 





HM-MED TEC 





CTR-A&R 








CTR-INTSIG 
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Source: Author Created from ROGER model output 











Table 23. ROGER Model Predicted Number of SRB Takers, FY08 


Average Planned 
Average Planned Graduates Graduates 
Baseline | Graduates Graduates Change from | Change from 
Predicted | Predicted Predicted Baseline Baseline 
Descriptor Takers (1) | Takers (2) Takers (3) (2-1) (3-1) 
IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 
ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 
HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author Created from ROGER model output 

































































Over the past few years, the managers of the ROGER model have been 
increasing the taker projections by certain arbitrary percentages to help account 
for some of the missed takers. Since FY06, the ROGER model taker predictions 
have been adjusted upwards by 30% or more across all ratings and NECs 


(except NUCs) to try to account for the under-predictions by the ROGER model. 


Table 24 gives the percentage increase in the number of predicted takers 
using the “modified” databases. This table allows for a comparison of the 
usefulness of the “modified” databases to the current method of using a fixed 
across-the-board percentage increase in predicted takers. Using the modified 
databases increased the predicted number of SRB takers by anywhere from 0 to 
400%, with the average increase in predicted takers for each fiscal year being 
between 40.9% and 51%. However, by dropping NEC 8425, which had a huge 
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percentage increase due to the low initial projections, the average percentage 


increase in predicted takers is between 23.3% and 30.4%. This data implies that 


using the modified databases are no more effective at increasing the number of 


predicted takers than by just applying the current fixed increase of around 30%. 


Table 24. 


Descriptor 
IT-SYS AD 


FY06 
Average 
Graduates 


and Baseline Models, FY06-08 


FY07 
Average 
Graduates 


FY07 
Planned 
Graduates 


FY08 
Average 
Graduates 


Percentage Change in Predicted Number of SRB Takers from Modified 


FY08 
Planned 
Graduates 





IT-NET SEC 





IT-ADV NET 





IS-IMAGE 





IS-NTCS 





5352 


SO-SEAL 
ND-SECOND 
SB-SWCC AD 





8425 


HM-SW IDC 





8432 


HM-PREV 





8452 


HM-XRAY AD 





8482 


HM-PHARM 





8483 


HM-SURG 





8506 


HM-MED TEC 





9138 


CTR-A&R 





9147 





CTR-INTSIG 





Average All 
Average Dropping 


2. 


NEC 8425 
Source: Author Created from ROGER model output 























Comparison of Number of Taker Predictions from ROGER to 
Actual Takers, FY06-FY08 


Tables 25-27 compare the actual takers to the predicted takers using the 
“modified” databases for FY06 to FY08. For the most part, the data shows that 


using the modified databases reduces the error between predicted takers and 


actual takers by a small amount. At the same time, for some NECs, there is an 


increase in error from using the modified databases. 
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The total percentage error change from the baseline taker predictions 


ranged from a reduction of 32.5 percentage points to an increase of 26.2 


percentage points, with the average being an error reduction of 8.6 percentage 


points. An 8.6 percentage point reduction in the error of missed takers seems to 


be minimal compared to the amount of work to compile and incorporate the 


CeTARS data into the EMR snapshot. There are five instances where the error 


increased (out of the 80 projections) and these are split across only three NECs. 


The increases in error from using the modified databases are shown in bold in 


Table 28 


Table 25. Comparison of Actual and Predicted Number of SRB Takers, FY06 


Descriptor 
IT-SYS AD 


FY06 
Actual 
SRB 
Takers (1) 


FY06 
Baseline 
Predicted 
Takers (2) 


FY06 
Average 
Graduates 
Predicted 
Takers 


(3) 


Baseline 


Actual: 
=(2-1) 


Average 
Graduates 
— Actual: 
=(3-1) 


Percent 
Error 
Using 

Baseline 


Percent 
Error 
Using 

Average 

Graduates 





IT-NET SEC 





IT-ADV NET 





IS-IMAGE 





IS-NTCS 





SO-SEAL 





ND-SECOND 





SB-SWCC AD 





HM-SW IDC 
HM-PREV 








HM-XRAY AD 
HM-PHARM 








HM-SURG 





HM-MED TEC 








CTR-A&R 
CTR-INTSIG 











Source: Author Created from ROGER model output 
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Table 26. Comparison of Actual and Predicted Number of SRB Takers, FY07 


FY07 FY07 Percent 
FY07 FY07 Average Planned Percent Error Percent 

Actual | Baseline | Graduates | Graduates Error Using Error Using 
SRB Minus Minus Minus Using Average Planned 

Descriptor Takers Actual Actual Actual Baseline | Graduates | Graduates 


IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 














ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 
HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author Created from ROGER model output 
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Table 27. 


Descriptor 
IT-SYS AD 


FY08 
Actual 
SRB 
Takers 


FY08 
Baseline 
Minus 
Actual 


FY08 
Average 
Graduates 
Minus 
Actual 


FY08 
Planned 
Graduates 
Minus 
Actual 


Percent 
Error 
Using 

Baseline 


Percent 
Error 
Using 
Average 
Graduates 


Comparison of Actual and Predicted Number of SRB Takers, FY08 


Percent 
Error 
Using 
Planned 
Graduates 





IT-NET SEC 





IT-ADV NET 





IS-IMAGE 





IS-NTCS 





SO-SEAL 
ND-SECOND 





SB-SWCC AD 





HM-SW IDC 





HM-PREV 





HM-XRAY AD 





HM-PHARM 
HM-SURG 





HM-MED TEC 
CTR-A&R 








CTR-INTSIG 














Source: Author Created from ROGER model output 
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Table 28. Change in Predicted Taker Error from Baseline Using Modified 
Databases, FY06-08 


FY06 FY07 FY07 FY08 FY08 
Average Average Planned Average Planned 
Descriptor Graduates | Graduates | Graduates | Graduates | Graduates 


IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 
ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 
HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author Created from ROGER model output 




































































While the total error reduction is important, the actual impact on the 
number of missed takers is the primary focus for this thesis. Tables 29-32 
display the percentage of missed takers that were accounted for by using the 
“modified” databases. The tables reveal a lot of variance in the ability of the 
“modified” databases to account for the missed takers, not only across each NEC 
but also across the fiscal years. Over half of the “modified” taker predictions 
accounted for less than 20% of the missed takers, while three-quarters of all the 
“modified” taker predictions accounted for less than 40% of the missed takers. 


While most of the NECs accounted for small percentages of the missed 
takers, some of the NECs show large percentage increases, but these large 
changes tend to be in those NECs with small numbers of missed takers using 
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the baseline database. One example is NEC 9147 in FY08, where the baseline 
predictions missed three takers and then the modified databases over-predicted 
by 20. 


Table 29. Percentage of Missed Takers Accounted For Using Modified Database, 


FY06 


FY06 Predicted 


Descriptor 


FY06 Missed 
Takers Using 
Baseline 


Taker Increase 
using Average 
Graduates 


Percent 
Missed 
Takers 


IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 
ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 
HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author Created from ROGER model output 
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Table 30. Percentage of Missed Takers Accounted For Using Modified Database, 
FYO7 


FY07 FY07 
FY07 Average Planned Average Planned 
Missed | Graduates | Graduates | Graduates | Graduates 
Takers Predicted | Predicted Percent Percent 
Using Taker Taker Missed Missed 
Descriptor Baseline | Increase Increase Takers Takers 


IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 
ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 


HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author Created from ROGER model output 
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Table 31. Percentage of Missed Takers Accounted For Using Modified Database, 
FY08 


FY08 FY08 
Average Planned Average Planned 

FY08 Graduates | Graduates | Graduates | Graduates 
Baseline | Predicted | Predicted Percent Percent 

Minus Taker Taker Missed Missed 
Descriptor Actual Increase Increase Takers Takers 


IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 
ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 


HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author Created from ROGER model output 
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Table 32. Compiled Percentage of Missed Takers Accounted for Using the Modified 
Databases, FY06-FY08 


FY06 FY07 FY07 FY08 FY08 
Average Average Planned Average Planned 
Descriptor Graduates | Graduates | Graduates | Graduates | Graduates 
IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 
ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 
HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author Created from ROGER model output 




































































C. BUDGET ANALYSIS 


As seen in Tables 33-35 using the modified databases increased the 
annual budget predictions across-the-board. The budget prediction for FY06 
increased by 20% using average graduates, while the budget prediction for FY07 
increased by 25% using average graduates and 27% using planned graduates. 
The budget prediction for FY08 increased by 17% using average graduates and 
18% using planned graduates. These are fairly large increases in the predicted 
budgets, but they do not seem to increase linearly with the increase in the 
predicted number of takers. This is because, while the average change in the 
predicted number of takers may have been in the 40 and 50 percentiles, this 
average increase was not the same for every NEC, and each NEC has different 
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costs per taker. Some of the high-cost NECs may have no increase in their 
budget costs, while other low-cost NECs may have a 200% increase from their 


original budget. 


Table 33. SRB Budget Predictions ($ Thousands), FY06 


Average 
Descriptor Baseline Graduates | Difference 
IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 
ND-SECOND 595 
SB-SWCC AD 744 
HM-SW IDC 95 
HM-PREV 320 
HM-XRAY AD 288 
HM-PHARM 61 
HM-SURG 409 
HM-MED TEC 567 
CTR-A&R 197 
CTR-INTSIG 
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69 
Total $8210 $9832 $1622 


Percentage Increase 
Over Baseline 20% 


Source: Author Created from ROGER model output 
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Table 34. 


Descriptor 
IT-SYS AD 


SRB Budget Predictions ($ Thousands), FY07 


Baseline 


Average 
Graduates 


Difference 
Average 
Graduates 
to Baseline 


Planned 
Graduates 


Difference 
Planned 
Graduates 
to 
Baseline 


Planned 
Graduates 
Minus 
Average 
Graduates 





IT-NET SEC 





IT-ADV NET 





IS-IMAGE 





IS-NTCS 





SO-SEAL 





ND-SECOND 





SB-SWCC AD 





HM-SW IDC 





HM-PREV 





HM-XRAY AD 
HM-PHARM 


93 





HM-SURG 


452 





HM-MED TEC 


873 





CTR-A&R 


262 





CTR-INTSIG 








Total 

















0 





40 


Percent Increase 
25% 27% 


Over Baseline 
Source: Author Created from ROGER model output 
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Table 35. SRB Budget Predictions ($ Thousands), FY08 


Difference Planned 
Average Difference | Graduates 
Graduates Planned Minus 
Average to Planned Graduates | Average 
Descriptor | Baseline | Graduates | Baseline | Graduates | to Baseline | Graduates 





IT-SYS AD $1835 $2136 $301 $2236 $401 $100 





IT-NET SEC 923 1227 304 1134 211 





IT-ADV NET 795 1083 288 1207 412 





IS-IMAGE 553 679 126 635 





IS-NTCS 1812 2103 291 2055 





SO-SEAL 8910 9143 233 9143 





ND-SECOND 1450 1769 319 1970 





SB-SWCC AD 1809 1825 16 1809 





HM-SW IDC 144 620 476 576 





HM-PREV 586 715 129 709 





HM-XRAY AD 209 564 355 523 





HM-PHARM 381 434 53 436 
HM-SURG 
HM-MED TEC 





CTR-A&R 























CTR-INTSIG -7 








Total 


Percentage Increase 
Over Baseline 17% 18% 


Source: Author Created from ROGER model output 





D. ANALYSIS OF INCREASING THE BASELINE NUMBER OF 
PREDICTED TAKERS BY A SET PERCENTAGE 


As stated previously, over the past few years the managers of the ROGER 
model have been increasing the taker projections by fixed percentages to help 
account for some of the missed takers. Since FY06, the ROGER model taker 
predictions have been adjusted upwards by 30% or more across all ratings and 
NECs (except NUCs) in order to try to account for the under-predictions by the 
ROGER model. 
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The data in Tables 36-38 show that the number of missed takers varies 
greatly across each NEC and fiscal year. This shows that applying a fixed 
percentage increase in taker predictions will not be effective in accounting for the 
missed takers. At the same time, using the CeTARS data in the ROGER model 
database had minimal effect upon accounting for the missed takers. By looking 
at Table 38, it is easy to see that for most NECs the predictions from the modified 
databases fell well short of being able to account for the missed takers in each 
fiscal year. While increasing the predicted number of takers may not give an 
exact accounting of the missed number of SRB takers, applying a set percentage 
increase across-the-board is easier, requires fewer man-hours, and is more cost- 


effective than incorporating the CeTARS data into the ROGER model databases. 


Increasing the predicted number of takers by an arbitrary fixed percentage 
does help increase the overall budget, but another more accurate method to 
account for the missed takers would be to determine the historical number of 
missed takers for each NEC and incorporate a percentage of these missed 
takers into the baseline predicted number of takers for each NEC. This seems a 
more accurate solution than using the databases modified with CeTARS 
graduation data or the current practice of increasing the ROGER model 
predictions by a fixed factor as it will allow for input of a percentage of the direct 
historical number of missed takers. Table 36 shows the large differences 
between the ROGER model’s baseline predicted number of SRB takers and the 
missed number of actual SRB takers. NECs 2780 is prime example of why 
applying a fixed percentage increase to predicted SRB takers is ineffective as its 
predicted takers were only 8, but the number of missed SRB takers was 179. 
Increasing the ROGER model taker predictions by a percentage of the missed 
taker numbers would be a closer fit than just increasing the number of predicted 


takers by a fixed percentage as it will be using a percentage of the actual error. 
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Table 36. Number of Baseline Predicted SRB Takers and Missed SRB Takers, 
FY06-FY08 


FY06 FY07 FY08 
Baseline Baseline Baseline 
Predicted Predicted Predicted 
Takers Takers Takers 


IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 
ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 
HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author Created from ROGER model output 
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Table 37. 


Missed Takers as a Percentage of Baseline Predicted Takers, FYO06-FY08 


Descriptor 
IT-SYS AD 
IT-NET SEC 
IT-ADV NET 
IS-IMAGE 
IS-NTCS 
SO-SEAL 
ND-SECOND 
SB-SWCC AD 
HM-SW IDC 
HM-PREV 
HM-XRAY AD 
HM-PHARM 
HM-SURG 
HM-MED TEC 
CTR-A&R 
CTR-INTSIG 
Source: Author Created from ROGER model output 


Average 
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Table 38. Percentage of Missed Takers Accounted for Using Modified Databases, 
FY06-FY08 


FY06 FY07 FY07 FY08 FY08 
Average Average Planned Average Planned 
Descriptor Graduates | Graduates Graduates Graduates Graduates 


IT-SYS AD 





IT-NET SEC 





IT-ADV NET 





IS-IMAGE 
IS-NTCS 





SO-SEAL 





ND-SECOND 





SB-SWCC AD 
HM-SW IDC 





HM-PREV 





HM-XRAY AD 





HM-PHARM 





HM-SURG 





HM-MED TEC 





CTR-A&R 























CTR-INTSIG 
Source: Author Created from ROGER model output 





E. CHAPTER SUMMARY 


Running the ROGER model using the databases modified with CeTARS 
graduation data caused an average increase in the eligibles pools from 25 to 
27%. At the same time, only 21 to 27% of the personnel assigned new NECs 
were selected as eligible by the ROGER model. The low eligibility selection of 
those who were assigned an NEC was in part due to the random assignment 
across the database in the pool of those personnel that qualified to obtain the 
NEC. Therefore, it is possible the NEC assignment method played a large role in 
the selection of the eligibles pool. Assigning the NECs to only those personnel 
who are in the reenlistment window should greatly increase the percentage of 
new NEC holders selected as eligible; however, at the same time, it may 
overstate the number who obtain an NEC and are SRB-eligible in the same year. 
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Using the modified databases did reduce the error in the number of 
missed takers by an average reduction of 8.6 percentage points. However, in a 
couple of cases, using the modified databases caused an in increase in the 
percentage of error for the missed takers. While using the modified databases 
reduced the overall error for missed takers, it showed no better ability to account 
for the missed takers than the current practice of increasing the predicted takers 
by a fixed percentage. In addition, it was hypothesized that the data on planned 
graduates would be better at accounting for the missed takers, but the data 
shows that there was little difference between the planned graduate data and the 


historical average graduate data. 


Using the modified databases increased the fiscal year predicted budgets 
from 17 to 27%, but the increases did not coincide with the increases in the 
average number of predicted takers. The difference between the overall 
percentage change in predicted budgets and the overall percentage change in 
predicted takers was caused from the varying effects of the modified databases 
on each NECs predicted takers and the differences in SRB costs for each NEC. 


This analysis used only 16 of the 116 NECs on the SRB eligibility list. It is 
possible that the small sample size used in this study skewed the results. 
Another more comprehensive study involving all of the NECs may show different 


results. 


Another option for accounting for the missed takers would be to determine 
each NEC’s historical number of missed takers and incorporate a percentage of 
these missed takers into each NEC’s baseline predicted takers. This seems a 
more viable solution than using the databases modified with CeTARS graduation 
data or the standard approach of increasing ROGER model predictions as it will 
allow for input of a percentage of the historical error of missed takers by NEC. 
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V. SUMMARY AND RECOMMENDATIONS 


A. CONCLUSIONS 


Over the past few years, the predictions of SRB takers have been 
adjusted upwards by an arbitrary fixed percentage to account for the missed 
takers in the ROGER model. These arbitrary adjustments have been applied 
across the entire database as a band-aid for the inaccuracies in the current SRB 
prediction model. One of the potential factors affecting the mis-predictions by the 
ROGER model is believed to be the acquisition of SRB-eligible NEC/skills during 
the projected fiscal year. The purpose of this thesis was to determine if 
incorporating NEC acquisition data into the ROGER models database would 


create more accurate taker predictions. 


While the average error in the number of predicted takers was reduced by 
8.6 percentage points, the effects varied greatly across NECs and did not show 
any improvement as compared to the current practice of increasing the predicted 
takers by a fixed percentage. Even so, this analysis used only a small portion of 
the NECs on the SRB list, which could have affected the overall results. Perhaps 
the analysis of a larger number of NECs from the SRB list would prove the 
modified database method to be more productive at reducing predicted taker 
error. There are still other potential factors affecting the ROGER model’s 


prediction ability, but these were beyond the scope of this thesis. 


B. RECOMMENDATIONS 


1. ROGER Model 


With the fluidity and differences in reenlistment policies across NECs, it is 
all but impossible to exactly predict the total number of SRB takers for a given 
year. The differences in reenlistment policies greatly affect the ROGER model’s 
ability to select all eligibles, which, in turn, affects the SRB-taker and budget 
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projections. This analysis showed some improvement in predictions by 
incorporating NEC/skill acquisition data into the ROGER model, but the overall 
effects were small and varied greatly across NECs. It is recommended that, in 
the short run, the managers of the ROGER model continue to use the fixed 
adjustment factor in taker projections to account for the missed taker error until a 


more precise solution is found. 


Another recommended option for accounting for the missed takers would 
be to determine the historical number of missed takers for each NEC, and 
incorporate a percentage of these missed takers into each NEC’s baseline 
number of predicted takers. This seems a potentially more viable solution than 
using the databases modified with CeTARS graduation data or the current 
practice of applying an arbitrary fixed factor, as it will allow for input of a 
percentage of the actual number of missed takers. The missed takers were 
shown in this study to be, in many cases, a few hundred to a few thousand 
percent of the baseline predicted takers. The difference in size of the predicted 
taker and missed taker numbers shows that incorporating a fixed percentage of 
the predicted takers does not accurately account for the actual missed takers. 


2. NETC CeTARS Database 


One of the major tasks in completing this thesis was the compilation of 
NEC skill acquisition data. The Navy does not currently have a database that 
compiles all of the NEC acquisitions that occur every fiscal year. The main 
historical database for Navy schools, the NETC CeTARS database, has some 
NECs assigned sporadically throughout the database, but the majority of the 
courses that award an NEC do not have the NEC assigned to them. This 
inconsistency led to much confusion during the data collection for this thesis and 
required the scouring of the database by each individual course code to ensure 
that all of the graduates for a given NEC were accounted for. If NETC is going to 
use NECs in the CeTARS database, the NECs should be assigned correctly and 
completely, otherwise it could lead to incorrect data usage in studies, such as 
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this one, that utilize NEC data. In addition, the SAG Corporation should be 
allowed remote access to the NETC CeTARS database to enable data access 
for follow-on studies involving NEC/skill acquisitions that are needed to improve 
the ROGER model predictions. 


3. Future Research 


The following suggestions are provided as recommendations for future 
research: 


e | recommend a more comprehensive analysis as a follow-up to this 
thesis that involves adding graduation data into the EMR snapshot 
across all NECs on the SRB list. 


e | recommend a similar study be completed with new NECs being 
assigned to only those individuals in the EMR snapshot who are 
eligible for reenlistment in that fiscal year. This would allow for 
analysis of the ROGER model's predictions of SRB-eligible numbers. 
This would determine if the low selection of eligibles during this 
analysis is due to the method of NEC assignment or if there is an 
inherent issue with the ROGER eligibility selection routine. 


e | recommend an analysis to determine, by NEC, the percentage of 
personnel who reenlist for an SRB in the same fiscal year they 
received a new NEC. This will allow for a correct percentage of 
graduates to be applied directly to the beginning fiscal year SRB 
predictions base. 
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